Jump to content

ISP Enabling and Usage

From RidgeRun Developer Wiki


Follow us on: YouTube Twitter LinkedIn Email Share this page

Share This Page

Qualcomm Dragonwing Technical Guide logo
Qualcomm Dragonwing Technical Guide

Qualcomm Dragonwing Technical Guide
Table of Contents [Sticky]



Problems running the pipelines shown on this page? Please see our GStreamer Debugging guide for help .

Introduction

These pipelines were tested in the Ubuntu downloaded sources (Ubuntu 24.04.4). ISP use was verified within the kernel logs running the pipelines in one terminal and the following command in another terminal: 

sudo dmesg -w | grep -E "CAM-ISP|cam_ife|IFE|CSID|IPP|RDI|BPS|ICP"

When running the pipeline, the following output confirms the use of the ISP: 

[  305.799457] CAM_INFO: CAM-ISP: __cam_isp_ctx_acquire_dev_in_available: 7816: Acquire success: session_hdl 0xbe0200 num_rsrces -16843010 ctx 1 link: 0xffffffff
[  305.799626] CAM_INFO: CAM-ICP: cam_icp_v2_download_fw: 873: Loading Secure PIL : Y
[  305.830225] CAM_INFO: CAM-ICP: cam_icp_mgr_hw_open: 5018: [icp] FW download done successfully
[  305.830236] CAM_INFO: CAM-ICP: cam_icp_mgr_process_dbg_buf: 2995: [icp]: FW_DBG:CICP_FW_E : HFI  :QC_IMAGE_VERSION_STRING=CICP.FW.7.8-00002,OEM_IMAGE_VERSION_STRING=CRM,BUILD_TIME: Apr 10 2025 23:00:32,CACHE_ENABLED at icphostinterface.c:880 QC_IMAGE_VERSION_STRING=CICP.FW.7.8-00002 OEM_IMAGE_VERSION_STRING=CRM
[  305.830242] CAM_INFO: CAM-ICP: cam_icp_mgr_process_dbg_buf: 2995: [icp]: FW_DBG:CICP_FW_E : HFI  :ELF variant: T480:USE_CDM_1_1:TCM_ENABLED: , API version: 0x2490000 at icphostinterface.c:881 QC_IMAGE_VERSION_STRING=CICP.FW.7.8-00002 OEM_IMAGE_VERSION_STRING=CRM
[  305.840482] CAM_INFO: CAM-ISP: cam_ife_mgr_v_acquire: 18742: is FE 0 Offline 0 ctx id 0
[  305.840567] CAM_INFO: CAM-ISP: cam_ife_hw_mgr_print_acquire_info: 1730: 14:27:13.237 Acquired Single IFE[1] with [9 pix] [0 pd] [1 rdi] ports for ctx:1 per_port_enabled :0
[  305.840593] CAM_INFO: CAM-ISP: __cam_isp_ctx_acquire_hw_v2: 8263: Acquire success: session_hdl 0xbe0200s ctx_type 3 ctx 1 link 0xa50306 hw_mgr_ctx 1 is_shdr 0 is_shdr_master 0 offline_isp :0
[  305.841711] CAM_WARN: CAM-ISP: __cam_isp_ctx_process_evt: 9076: Get unexpect evt:2 in acquired state, ctx: 1 on link: 0xa50306
[  305.843255] CAM_INFO: CAM-CSIPHY: cam_csiphy_core_cfg: 2865: CAM_START_PHYDEV: 0, CSID:1, Type: DPHY, dev_cnt: 1, slot: 0, combo: 0, cphy+dphy: 0, skew_en: 0, sec_mode: 0, Datarate: 720000000, Settletime: 3200000000

ISP GStreamer Examples

The following table shows some of the qtiqmmfsrc element properties that can be tuned. See complete table in Qualcomm's Documentation.[1]

Property Description Values / Range
control-mode Top-level 3A control switch: auto-exposure, auto-white-balance, auto-focus. off, auto, use-scene-mode, off-keep-state
effect Effect applied to camera frames. off, mono, negative, solarize, sepia, posterize, whiteboard, blackboard, aqua
scene Camera optimizations for a scene. disabled, face-priority, action, portrait, landscape, night, night-portrait, theatre, beach, snow, sunset, steady-photo, fireworks, sports, party, candlelight, hdr
antibanding Adjusts exposure to avoid illumination banding. off, 50 Hz, 60 Hz, auto
sharpness Image sharpness strength. 0–6
contrast Image contrast strength. 1–10
saturation Image saturation strength. 0–10
iso-mode ISO exposure mode. auto, deblur, 100, 200, 400, 800, 1600, 3200, manual
manual-iso-value Manual ISO value used when iso-mode=manual. 100–3200
exposure-mode Camera exposure routine mode. off, auto
exposure-lock Locks current exposure values. true, false
exposure-metering Exposure metering mode. average, center-weighted, spot, custom
exposure-compensation Adjusts target brightness in steps. -12–12
manual-exposure-time Manual exposure time in nanoseconds, used when exposure-mode=off. 0–9223372036854775807
custom-exposure-table Custom exposure table structure. GstStructure
white-balance-mode White balance routine mode. off, manual-cc-temp, manual-rgb-gains, auto, shade, incandescent, fluorescent, warm-fluorescent, daylight, cloudy-daylight, twilight
white-balance-lock Locks current white balance values in non-manual modes. true, false
manual-wb-settings Manual white balance settings: color temperature or RGB gains. GstStructure
focus-mode Auto-focus mode. off, auto, macro, continuous, edof
noise-reduction Noise reduction filter mode. off, fast, hq
noise-reduction-tuning Noise reduction tuning structure. GstStructure
zoom Camera zoom rectangle in sensor active pixel array coordinates. <X, Y, WIDTH, HEIGHT>

Simple camera display

gst-launch-1.0 -e qtiqmmfsrc camera=0 ! video/x-raw,format=NV12,width=1280,height=720,framerate=30/1,interlace-mode=progressive,colorimetry=bt601 ! waylandsink

Adjusted camera

gst-launch-1.0 -e qtiqmmfsrc camera=0 white-balance-mode=6 focus-mode=1 iso-mode=0 exposure-mode=0 saturation=5 manual-exposure-time=43333333 ! video/x-raw,format=NV12,width=1280,height=720,framerate=30/1,interlace-mode=progressive,colorimetry=bt601 ! waylandsink

Purpose:

  • Validate live video preview
  • Confirm camera bring-up
  • Verify NV12 output
  • Configure camera to white-balance-mode=6 (fluorescent), focus-mode=1 (auto), iso-mode=0 (auto), exposure-mode=0 (off), saturation=5 (default), manual-exposure-time=43333333

Camera with max saturation

gst-launch-1.0 -v -e qtiqmmfsrc camera=0 white-balance-mode=6 focus-mode=1 iso-mode=0 exposure-mode=0 manual-exposure-time=43333333 saturation=10 ! video/x-raw,format=NV12,width=1920,height=1080,framerate=30/1 ! queue max-size-buffers=4 leaky=downstream ! qtivtransform ! perf print-cpu-load=1 ! waylandsink

Purpose:

  • Change image saturation, by default is 5

Before

After

Camera with max contrast

gst-launch-1.0 -v -e qtiqmmfsrc camera=0 white-balance-mode=6 focus-mode=1 iso-mode=0 exposure-mode=0 saturation=5 manual-exposure-time=43333333 contrast=10 ! video/x-raw,format=NV12,width=1920,height=1080,framerate=30/1 ! queue max-size-buffers=4 leaky=downstream ! qtivtransform ! perf print-cpu-load=1 ! waylandsink

Purpose:

  • Change image contrast, by default is 5

Before

After

Camera with incandescent white balance mode

gst-launch-1.0 -v -e qtiqmmfsrc camera=0 focus-mode=1 iso-mode=0 exposure-mode=0 saturation=5 manual-exposure-time=43333333 white-balance-mode=5 ! video/x-raw,format=NV12,width=1920,height=1080,framerate=30/1 ! queue max-size-buffers=4 leaky=downstream ! qtivtransform ! perf print-cpu-load=1 ! waylandsink

Purpose:

  • Change white balance mode from 6 (fluorescent) to 5 (incandescent)

Before

After

FPS / Performance Display

gst-launch-1.0 -e qtiqmmfsrc camera=0 white-balance-mode=6 focus-mode=1 saturation=5 iso-mode=0 exposure-mode=0 manual-exposure-time=43333333 ! video/x-raw,format=NV12,width=1280,height=720,framerate=30/1,interlace-mode=progressive,colorimetry=bt601 ! fpsdisplaysink video-sink=waylandsink text-overlay=true sync=false

Purpose:

  • Measure frame rate
  • Detect frame drops
  • Validate throughput

Fakesink Test

gst-launch-1.0 -e qtiqmmfsrc name=camsrc camera=0 ! video/x-raw,format=NV12,width=1280,height=720,framerate=30/1,interlace-mode=progressive,colorimetry=bt601 ! fakesink sync=false

Purpose:

  • Test video without display
  • Useful for SSH and performance measurements

Camera to H.264 Recording

gst-launch-1.0 -e qtiqmmfsrc name=camsrc camera=0 ! video/x-raw,format=NV12,width=1280,height=720,framerate=30/1,interlace-mode=progressive,colorimetry=bt601 ! v4l2h264enc capture-io-mode=4 output-io-mode=5 extra-controls="controls,video_bitrate=6000000,video_bitrate_mode=0;" ! h264parse ! mp4mux ! filesink location=camera-720p30.mp4

Purpose:

  • Validate Camera source → VPU encode path
  • Record video to MP4

Camera to H.265 Recording

gst-launch-1.0 -e qtiqmmfsrc name=camsrc camera=0 ! video/x-raw,format=NV12,width=1280,height=720,framerate=30/1,interlace-mode=progressive,colorimetry=bt601 ! v4l2h265enc capture-io-mode=4 output-io-mode=5 extra-controls="controls,video_bitrate=6000000,video_bitrate_mode=0;" ! h265parse ! mp4mux ! filesink location=camera-720p30-h265.mp4

Purpose:

  • Validate HEVC encoding path

Camera source with Qualcomm Transform

gst-launch-1.0 -e qtiqmmfsrc name=camsrc camera=0 ! video/x-raw,format=NV12,width=1280,height=720,framerate=30/1,interlace-mode=progressive,colorimetry=bt601 ! qtivtransform flip-horizontal=true ! waylandsink

Purpose:

  • Validate ISP → QTI processing
  • Check hardware transform path, it shows image flipped

Before

After


Camera with Split (Preview + Record)

gst-launch-1.0 -e qtiqmmfsrc name=camsrc camera=0 ! video/x-raw,width=1280,height=720 ! tee name=t t. ! queue ! waylandsink t. ! queue ! v4l2h264enc capture-io-mode=4 output-io-mode=5 extra-controls="controls,video_bitrate=6000000,video_bitrate_mode=0;" ! h264parse ! mp4mux ! filesink location=camera-preview-record.mp4

Purpose:

  • Simultaneous preview and recording
  • Validate bandwidth and buffering

Camera with Metadata Extraction

gst-launch-1.0 -e qtiqmmfsrc name=camsrc camera=0 ! video/x-raw,width=1280,height=720 ! qtimlmetaextractor ! fakesink sync=false

Purpose:

  • Validate metadata flow with GST_DEBUG=qtimlmetaextractor:5

Performance measurements

Camera simple measurements

The following table shows baseline camera performance using qtiqmmfsrc with NV12 format and no additional processing. GPU usage is not applicable in this configuration. 

gst-launch-1.0 -e qtiqmmfsrc camera=0 ! video/x-raw,format=NV12,width=$WIDTH,height=$HEIGHT,framerate=30/1 ! perf print-cpu-load=1 ! fakesink sync=false
Resolution %MEM RSS (KB) Mean FPS CPU (%)
1280x720 0.1 67652 29.948 12
1920x1080 0.1 67596 29.939 13
3840x2160 0.1 67584 29.947 13

Camera with qtivtransform and leaky queue

This table shows camera performance when adding a leaky queue and qtivtransform. This configuration introduces buffering and transformation but no GPU-based processing. This is the pipeline used for testing 

gst-launch-1.0 -e qtiqmmfsrc camera=0 ! video/x-raw,format=NV12,width=$WIDTH,height=$HEIGHT,framerate=30/1 ! queue max-size-buffers=4 leaky=downstream ! qtivtransform ! perf print-cpu-load=1 ! fakesink sync=false
Resolution %MEM RSS (KB) Mean FPS CPU (%)
1280x720 0.2 103476 29.939 12
1920x1080 0.2 103492 29.923 13
3840x2160 0.2 103428 29.882 13


Qualcomm Dragonwing Technical Guide logo
Qualcomm Dragonwing Technical Guide
Back to top ↑

  1. ↑ https://docs.qualcomm.com/doc/80-70018-50/topic/qtiqmmfsrc.html
Retrieved from "https://developer.ridgerun.com/wiki/index.php?title=Qualcomm_Dragonwing_Technical_Guide/Multimedia_Support/Acceleration/ISP_Enabling_and_Usage&oldid=84143"
Cookies help us deliver our services. By using our services, you agree to our use of cookies.